Handling occlusion in object tracking in stereoscopic video sequences

Stereo vision is a part of the field of computer vision that uses two or more cameras to obtain two or more images from which distance information can be obtained by using the triangulation principle [1]. We can obtain three-dimensional information of static scenes by using this technique. On the other hand, we can use optical flow algorithms for object tracking in two dimensional video sequences along the time [2]. In previous works [3], we have studied a system that combines stereoscopic vision and optical flow algorithms for object tracking in a three-dimensional space. We select a set of points to be tracked in the first frame of one video of the stereo sequence and, by using optical flow algorithms, we track them in that sequence. For each frame, we calculate the disparity of these points by using the other video and stereo algorithms. In this way, we know the position of each point in a three dimensional space (disparity) and along the time (optical flow). One of the most important problems of this technique is that this method is not able to handle the occlusion of the moving objects. For solving this occlusion problem, we propose the use of adaptive filters [4, 5] and neural networks [6] to predict the expected 3D velocities of the objects. The use of adaptative filters and neural networks for handling occlusion has been tested successfully in optical flow algorithms for 2D object tracking [7]. In this paper, we will analyze the use of these prediction techniques in a 3D system, by combining adaptative filters and neural networks, optical flow algorithms and stereo vision, in order to handle occlusion in object tracking in stereoscopic video sequences.